MATH 561: Numerical Analysis I

Problem 1 (Convergence of Richardson iteration). This method’s operator has the form T = I − BA = I − ωA because B = ωI. The eigenvectors of T are – as is easy to verify, the same as those of A, and the eigenvalues of T are λi(T ) = 1− ωλi(A). To prove that the method converges, it is convenient to verify that the l2 norm of T is less than one because we can then express everything in terms of the eigenvalues of T . Recall that the l2 norm of a square, invertible matrix equals its largest eigenvalue by magnitude. Now, since A is positive definite, all of its eigenvalues are positive. As a consequence, with ω > 0, the eigenvalues of T all satisfy λi(T ) = 1 − ωλi(A) < 1, but without a condition on ω, they can be arbitrarily negative. To make sure that they are less than one by magnitude, we need to require that ω < 2/λmax(A) since then −1 < λi(T ) = 1 − ωλi(A) < 1, and consequently ‖T‖l2 < 1. This condition ensures convergence of the Richardson method. In practice, one often finds that as problems grow, the largest eigenvalues of matrices also grow. This means that we will need to choose ω smaller and smaller as problems grow larger and larger, and in applications this typically leads to slower and slower convergence of the method. (Recall that if we set ω = 0, then the iteration stays where it is and makes no progress at all.) Richardson’s method is therefore more of theoretical interest and not typically used in practice.